Individual differences in response times

.title[
# Individual differences in response times
]
.author[
### Adriana F. Chávez De la Peña
]
.date[
### April 10, 2023
]

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

**1.- The study of individual differences.**

We discuss the distinctions between experimental and differential psychology, with an emphasis on how the different ways in which they treat between-subject variability leads to a *reliability paradox*.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

**2.- Individual differences in cognitive control.**

We discuss the specific case of cognitive control, where correlations across dependent measures obtained through experimental paradigms have been found to be weak.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

**3.- Response time distribution analysis**

We suggest a shift from correlational studies onto response time distribution analysis. We review models commonly used when working with response time (RT) data, distinguishing between simple RT models and choice-RT models.

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

**4.- Cognitive latent variable modeling of response times**

We talk about cognitive psychometrics and cognitive latent variable models (CLVM) as a sophisticated tool with the potential to unveil the underlying structure of parameters retrieved from RT models.

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

**5.- Research proposals**

---

# 1. The study of individual differences

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:**

- **Q:** How does performance change between *groups*? 
  
--
  
- ANOVA-based

- Experimental design aims to minimize between-subjects variance in order to attribute the results observed to the manipulations imposed by the experimenter only.

**Differential research:**

- **Q:** How does performance change between *individuals*?

- Correlation-based.

- Individual-level traits are captured by the correlations between measurements taken across subjects.

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:**

- **Q:** How does performance change between *groups*?

- ANOVA-based

- **Experimental design aims to minimize between-subjects variance in order to attribute the results observed to the manipulations imposed by the experimenter only.**

**Differential research:**

- **Q:** How does performance change between *individuals*?

- Correlation-based.

- **Individual-level traits are captured by the correlations between measurements taken across subjects.**

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:**

- **Q:** How does performance change between *groups*?

- ANOVA-based

- **Reduce between-subject variance**

**Differential research:**

- **Q:** How does performance change between *individuals*?

- Correlation-based.

- **Maximize between-subjects variance**

---

## Reliability

- We talk about reliability to refer to the degree to which we can reliably obtain the same results under similar conditions.

- How *reliable* are our findings?

- Typically quantified in terms of the correlation between related measures (i.e., parallel measures; Lord et al., 1968).

---

## Reliability

- An experimental effect is reliable to the extent that similar findings are reported consistently across replicate studies.

- A measurement is reliable as long as it consistently rank-orders individuals across applications.

The closer the performance observed across participants...

… the better, for an experimental researcher.

… the worse, for a differential researcher.

**Robust experimental effects do not necessarily imply reliable individual differences.**

---

### An example by Logie et al. (1996)

A study to explore the reliability of the **word-length effect** and the **phonological-similarity effect** on a serial order recall task.

**First experiment**  ( `$N=251$` )

- Strong aggregate effects (group mean differences),

**Retest ** ( `$N=40$` )

- Again, strong aggregate effects.

- Low correlations (r = 0.1 - 0.3) across experiments.

**Whether a subject showed the effect in the initial test was not predictive of their performance on retest.**

---

## Reliability paradox (Hedge et al., 2018)

- Experimental tasks minimize between-subject variability, which restrains the correlations one can compute from these measures.

- Robust experimental effects do not necessarily imply reliable individual differences.

- Low correlations found between related task-derived measures present a challenge for differential researchers.

---

# 2. Individual differences in cognitive control

---

# Cognitive control

The ability to ignore irrelevant information and to suppress prepotent responses when solving a task (Logan & Cowan, 1984).

**EXAMPLE: STROOP TASK**

- YELLOW

- GREEN

- RED

- BLUE

**Two experimental conditions:** Congruent / Incongruent

---

# Cognitive control

There are three main executive functions (Miyake et al., 2000):

- Shifting between mental sets or tasks. (*Shifting*).

- Controlling inputs and responses considered when solving a task. (*Cognitive control*).

- Monitoring and updating the contents of working memory. (*Updating*)

Cognitive control has two main components (Friedman & Miyake, 2004):

- Response-distracter inhibition

- Resistance to proactive interference

---

## Cognitive control tasks

### Flanker task

---

## Cognitive control tasks

### Global/local task

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/globalLocal_incongruent.png" alt="Andres and Fernandes, 2006" width="50%" />
Andres and Fernandes, 2006
</div>

---

## Cognitive control tasks

### Simon task

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/simonnnnn.png" alt="Kharitonova et al. (2013)" width="95%" />
Kharitonova et al. (2013)
</div>

---

## Cognitive control tasks

### Antisaccade

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/antisaccade.png" alt="Everling and Fischer (1998)" width="50%" />
Everling and Fischer (1998)
</div>

---

## Cognitive control tasks

Common elements...

- Distinction between “congruent” and "incongruent" trials.

- The core assumption is that participants should take longer to respond to incongruent trials.

- Cognitive control is measured in terms of differences in response times (RT) observed across conditions following **Donders' subtraction method**

- Data cleaning often involves removing extreme RTs and trials where an error was made.

---

## Donders' subtraction method (1868)

- Mental processes do not occur simultaneously.

- Distinctive mental processes can be isolated by subtracting the **mean response time** observed across different conditions.

---

## Individual differences in cognitive control

- What can we learn about one person's cognitive control from any given cognitive control task?

- Can we predict performance on task B from the score observed on task A?

- What are the **individual differences in cognitive control**?

---

## Individual differences in cognitive control

(Rey-Mermet et al., 2019)

---

## Individual differences in cognitive control

---

## Individual differences in cognitive control

---

## Big debate: What do these low correlation imply?

**1.- A substantive problem:** Is cognitive control a cognitive process that can be studied as an individual trait or not?

**2.- A measurement challenge:** Does it make sense to use difference scores to study individual differences in cognitive control?

---

## The measurement challenge

Correlational studies are conducted to capture individual differences in a given trait of interest under two core assumptions:

- 1) We are using scores related to the same process or to different components of a unitarian process.

- 2) We are using scores that provide reliable information.

**Task-specific difference scores need to be highly reliable.**

---

## Reliability of difference scores

- Hedge et al. (2018) conducted three test-retest studies. The most reliable cognitive control task was found to be the Stroop task ( `$0.60 - 0.66$` )

- Enkavi et al. (2019) summarize reliability indices reported in the literature, which were found to be highly variable (i.e., the Stroop task  varied between 0.27 and 0.8, and the Simon task, oscillated between the whole `$0-1$` interval).

---

## Reliability of difference scores

`$$\Huge \rho_{dd'} = \frac{\rho_{xx'}-\rho_{xy}}{1-\rho_{xy}}$$`

Lord (1963), cited by Draheim et al. (2019)

Let's assume...

...Individual components have perfect reliability ( `$\large \rho_{xx'}=1$` )

...Individual components are independent from each other ( `$\large \rho_{xy}=0$` )

---

## Reliability of difference scores

- High correlations between incongruent and congruent RT are expected.

- It is expected to have shared systematic variance across incongruent and congruent RTs.

- Subtracting one of them from the other removes the systematic variance, leaving only the error variance.

---

## Factorial structure of overall RT

**Rey-Mermet et al., 2019**

---

## Factorial structure of difference scores

**Rey-Mermet et al., 2019**

---

## Arguments against difference scores

- “Differences between scores are much more unreliable than the scores themselves” (Lord, 1956).

- “Scores formed by subtracting pretest scores from postest scores lead to fallacious conclusions, primarily because such scores are systematically related to any random error of measurement” (Cronbach and Furby, 1970)

- “Difference scores have low convergent validity caused by deficiencies in test-retest reliability” (Paap & Sawi, 2016)

**Overall:** Difference scores are particularly problematic for differential research.

---

## What are the alternatives?

--

- 1) Switching to accuracy metrics.

- 2) To develop a new metric that incorporates accuracy and RTs.

- 3) A different modeling approach...

---

# 3. Response time modeling

---

##  Response time (RT) modeling

- RTs are known to be highly variable.

- The analysis of RT distributions constitutes a long-standing tradition in Psychology (Hohle, 1965; Luce, 1986; Ratcliff, 1978)

- Specific characteristics of RT distributions are thought to shed light on the psychological processes that underlie the behavior observed.

---

## The Worst Performance rule

- Slowest RTs are more indicative of general intelligence than the faster RTs (Larson & Alderton, 1990).

- When individual RTs are ranked and grouped into RT bands, the median slowest RTs correlate better with measures of general intelligence, working memory and processing speed than the fastest median RTs (Coyle, 2003).

- Point estimates such as the mean or median RT neglects the RT variability observed.

---

## Two types of RT models

**1.- Simple RT models**

Mathematical models that describe empirical RT distributions and whose quantitative properties (i.e., shift, scale and shape parameters) can be mapped to specific experimental manipulations (Andrews & Heathcote, 2001; Heathcote et al., 1991; Hockley, 1984; Hohle, 1965; Spieler et al., 2000).

**2.- Choice RT models**

RTs are conceived as the stopping point of an internal processes generating the choices observed, (Noorani & Carpenter, 2016), and as such, RTs are modeled jointly with the choices observed.

---

## Simple RT models: the exGaussian distribution

---

## Simple RT models: the exGaussian distribution

- The exGaussian distribution has proven suitable to model empirical RT distributions (Heathcote et al., 1991; Hockley, 1984; Mewhort et al., 1992; Ratcliff, 1979).

- Parameters related to the normal component have been associated with peripheral sensory–motor and automatic processes

- The exponential component has been found to be related to decision-related processes (Hohle, 1965; Gordon & Carson, 1990; Rohrer & Wixted, 1994).

- The exGaussian parameters have also been mapped to drift diffusion model parameters (e.g., faster drift rates are associated with smaller rates, while more conservative response criteria are linked with larger values of `$\mu$` ; Spieler, 2001; Schmiedek et al., 2007).

---

## Choice RT models

“...most real-life decisions are composed of two separate decisions: first the decision to stop deliberating and act, and then the decision or act itself”.

Forstmann et al. (2016)

- Sequential sampling models account for the speed-accuracy tradeoff observed (Stone, 1960; Ratcliff & Smith, 2004).

- These sampling models assume that evidence for any response alternative is accumulated over time until a response threshold is reached.

- They account jointly for the responses and RTs observed.

---

### LATER model (Carpenter, 1981)

- The linear approach to threshold with ergodic rate (LATER) model considers only **two free parameters**.

- Founded on the empirical finding that the inverse of the RTs follow a normal distribution (with parameters `$\mu$` and `$\sigma$` ).

- Non-decision time is assumed to be fixed and is otherwise not considered by this model.

---

### LATER model

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/LATER.png" alt="Noorani and Carpenter, 2016" width="90%" />
Noorani and Carpenter, 2016
</div>

---

### Linear ballistic accumulation model (Brown & Heathcote, 2008)

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/LBA.png" alt="Brown and Heathcote, 2008" width="98%" />
Brown and Heathcote, 2008
</div>

---

### Drift diffusion model (Ratcliff, 1978)

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/DDM_Illustration3.png" alt="Johnson et al., 2017" width="90%" />
Johnson et al., 2017
</div>

---

### Circular drift diffusion model (Smith, 2016)

---

# 4. Cognitive latent variable modeling of response times

---

## Latent variable models and psychometrics

- Cognitive models account for the psychological processes that underlie the data observed.

- Psychometric models (LVMs) assumes that the data observed can be explained as the linear combination of latent variables that vary across different levels (i.e., across participants, conditions, tasks, etc.; Baribault, 2019).

- LVMs do not assume that every latent variable has a relevant psychological interpretation (Bollen, 2002).

---

## Cognitive psychometrics and individual differences

- Cognitive psychometrics refers to the psychometric treatment of cognitive models (Batchelder, 1998; Batchelder, 2010).

- The goal is to identify whether the cognitive processes described by cognitive models vary across participants, items, conditions, tasks, and so on.

- Cognitive psychometrics reconciliates the differences between experimental and differential research (Cronbach, 1957; Borsboom, 2006).

---

## Cognitive psychometrics and individual differences

Some examples:

- Schmiedek et al. (2007) fit an exGaussian model to the RT data and parameters of the DDM were estimated using both RTs and choice data. Then, the authors performed CFA techniques on all parameter values obtained.

- Studies on the latent variable structure of cognitive control from the perspective of cognitive psychometrics (Miyake et al., 2000; Friedman and Miyake, 2004).

---

## Cognitive latent variable models

- Vandekerckhove (2014) introduced a class of models that explore the latent variable structure of the parameters of cognitive models, all as part of a hierarchical Bayesian model.

- This feature comes with the huge advantage that the uncertainty contained in the data is captured and allocated across both model components (Vandekerckhove, 2014; Baribault, 2019).

---

# 5. Research proposals

---

## 1) Exploring the unidimensionality of processing speed

**Goal:** Explore the dimensionality of processing speed in extended data sets, in contrast with the dimensionality of human anthropometrics.

- Perform a PCA over the processing speed observed across different tasks.

- Perform a PCA over a large anthropometric dataset.

---

## 2) exGaussian CLVM

**Goal:** To explore the unidimensional structure of processing speed across cognitive control tasks through the lens of RT distribution analysis.

- Determine whether this univariance can be tracked back to the shift, scale, or shape parameters of the RT distribution.

---

## 3) Categorical drift-diffusion model

*Goal:* To develop extension of the CDDM where the bounded continuous decision space is segmented into bins.

- Develop a JAGS module.

- Present a paremet recovery study.

- Show an application to real-world data.

---

# IN SUMMARY

1.- The study of individual differences depends on high between-subject variability.

2.- Individual differences in cognitive control have been mostly approximated through task-derived dependent measures.

3.- Response Time distribution analysis presents an alternative to the use of task-derived dependent measures.

4.- We can implement cognitive latent variable models that capture the latent structure of the parameters of a given RT model.

---

# Thank you!